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(57) Abstract 

Apparatus and a related method for managing entities in a 
complex and, in general, geographically distributed system, such 
a distributed data processing system. The management approach 
is defined in terms of a generalized model having management 
modules integrated into a single cooperative system by a manage- 
ment director kernel. The managment modules include presenta- 
tion modules to provide an interface with users who manage the 
complex system, access modules to provide an interface with ma- 
naged entities or devices, and function modules to define various > 
functions that may be performed in controlling or monitoring the 
managed entities. If the complex system being managed is large, 
a managed entity and an associated access module may be locat- 
ed on one physical system, while a presentation module is located 
on another physical system, close to the user, and a function mo- 
dule being used might be located on yet another physical system, 
for reasons of processing convenience. The present invention pro- 
vides a convenient mechanism, consistent with the management 
model, for forwarding procedure calls between management mo- 
dules located on different physical systems, through management 
director kernels located on different physical systems. Two types 
of remote procedure calls are disclosed, one to forward procedure 
calls for invoking primitive functions, each on a single managed 
entity, and a more powerful remote procedure call for invoking 
higher-level functions relating to user-defined domains of multi- 
ple managed entities. 



SYSTEM -*1 



22tt) 



SYSTEM #2 



40AC1). 



42A( 1 ). 



INFO. 
MANAGER 



DISPATCHER 




T-40AC2) 



7-42A(2> 



U(2> 



SYSTEM #3 



INFO. 



MANAGER ^OBO) 



DISPATCHER X42B<3) 



I 







FOR THE PURPOSES OF INFORMATION ONLY 








Codes used 10 identify States pany to the PCT on the front pages of pamphlets publishing international 


applications under the PCT. 










AT 


Austria 


PR 


France 


MR 


Mauritania 


AU 


Australia 


GA 


Gabon 


MW 


Malawi 


BB 


Barbados 


GB 


Untied Kingdom 


NL 


Netherlands 


BE 


Belgium 


GN 


Guinea 


NO 


Norway 


BP 


Burkina Faso 


GR 


Greece 


NZ 


New Zealand 


BG 


Bulgaria 


HU 


Hungary 


PL 


Poland 


BJ 


Benin 


IE 


Ireland 


PT 


Portugal 


BR 


Brazil 


IT 


Italy 


RO 


Romania 


CA 


Canada 


JP 


Japan 


RU 


Russian Federation 


CF 


Central African Republic 


KP 


Democratic People** Republic 


SO 


Sudan 


CC 


Congo 




of Korea 


SE 


Sweden 


CH 


Switzerland 


KR 


Republic or Korea 


SK 


Slovak Republic 


CI 


t'dlcd'lvairt: 


KZ 


Kazakhstan 


SN 


Senegal 


CM. 


Cameroon 


I.I 


Liechtenstein 


SU 


Soviet Union 


CS 


Orchoslova j 


LK 


Sri L^inka 


TD 


Chad 


C2 


C^uxli Rcpublic 


IJJ 


(.uxemnourg 


TC 


Togo 


DE 


Uermany 


MC 


Monaco 


UA 


Ukraine 


DK 


Denmark 


MC 


Madagascar 


US 


Untied Stales of America 


ES 


Spain 


Ml. 


Mali 


VN 


Vict Nam 


Pi 


Finland 


MN 


Mongolia 







WO 93/20508 



PCT/US93/03402 



5 



10 

ENTITY MANAGEMENT SYSTEM WITH REMOTE CALL FEATURE 



15 BACKGROUND OF THE INVENTION 

This invention relates generally to the field of management 
of complex systems, such as distributed digital data process- 
ing systems, and, more particularly, to the management of 
20 complex systems that are geographically dispersed. Even more 
specifically, one aspect of the present invention relates to 
a technique that facilitates remote management of logical 
groupings of "target entities," i.e. targets of a management 
operation. 

25 

As digital data processing systems, or computers, 
have become smaller and less expensive, individual computers 
are being used by individuals and small groups. To enhance 
sharing of data, communication among users, and economy in 
30 connection with computing resources, computers have been 
connected into networks, over which communication takes place 
by means of messages transmitted over communication links . In 
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addition to the user computers, networks include "servers/ 
which provide services to users connected to the networks . A 
server may, for example, may control access to large amounts 
of data, or may control printers, telecommunications equip- 
5 ment, and so forth. A server may also provide specialized 
computational services, such as database searching and 
sorting. The servers and the user computers, sometimes 
referred to as clients, are interconnected by communications 
links to permit messages to be transferred among various 
10 computers and servers , and all of these components comprise 
a distributed system. 

Management of such complex distributed systems 
encompasses at least five functional areas: configuration 
management, fault management, performance management, 
15 accounting management and security management. Configuration 
management includes the ability to modify operating parameters 
of a network and its components, and the ability to identify 
every component and to reconfigure the network. Fault 
management refers to the detection, diagnosis, correction , 
20 and prevention of network and system faults and error 
conditions. Performance management is the monitoring of the 
performance of managed "objects" or "entities, " as well as the 
monitoring of the network as a whole. Accounting management 
deals with the ability to monitor the use of network and 
25 system resources, to identify the costs associated with usage, 
and to generate the data needed to charge the appropriate 
users. Security management defines those facilities required 
to manage services, such as authentication of users 1 and 
providers' identities, control of access to resources, and the 
30 confidentiality of information within the network environment . 

More generally, network management functions fall 
into just two categories: monitoring and control. A managed 
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"entity" may be subject to control by a management function, 
such as when a network element is activated or switched to a 
different operating mode or speed, or may be subject to 
monitoring, such as when a computer device is polled for. its 
5 current status. Management functions are rendered more complex 
by the diverse nature of many networks . A distributed network 
is likely to combine voice data, text data and image technolo- 
gies, is likely to contain products supplied by multiple 
vendors, and is, by definition, widely -distributed geographi- 

10 cally. In addition, the network may be growing rapidly and 
consist of thousands of computer systems. 

The management process for controlling or monitoring 
a single managed entity can be conveniently viewed is 
involving two components: the managed entity arid a managing 

15 entity, referred to in this specification as a director. The 
director includes a presentation module, for interface with 
elements outside of the director, such as with management 
personnel; a function module, which defines one or more 
* management functions that can best be characterized as higher- 

20 level or value-added functions; and an access module, defining 
mechanisms for control and monitoring of a managed entity at 
a lower level. In contrast to the higher-level functions 
performed by function modules, access modules perform 
functions at what is best characterized as a primitive level. 

25 Practically all problems involving the management of complex 
systems can be analyzed using this management director model. 
For directors that control or monitor multiple managed 
entities, there may be multiple presentation modules, multiple 
function modules and multiple- access modules, although the 

30 director may have a common interface for connecting the 
various modules, and a common executive. Moreover, there may 
be sharing of some modules. For example an access module 
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providing access to a particular managed entity may be shared 
by multiple function modules. Specifically, a single access 
module might provide access to a computer system for purposes 
of performance monitoring (involving a first function module) 
5 and modification of operating parameters (involving a second 
function module) - 

A potential difficulty in entity management is that 
the necessary modules of the director may be geographically 
dispersed. The access module must often be located close to 

10 the managed entity, the presentation module is typically 
located close to the manager or operator, but may be located 
on a different local area network, and the function module may 
well be located on a third computer system. 

The need for geographic dispersion of the management 

15 modules will not always arise. For example, in a single self- 
contained network with perhaps only a few dozen components, 
a single management system will suffice to allow a single 
manager to control and monitor the environment. If the 
distributed network is spread over a small number of intercon- 

20 nected sites, separate site managers might well be able to 
handle most of the management problems r with occasional face- 
to-face meetings to resolve any issues that require their 
mutual cooperation. Again there is no need for a distributed 
solution. However, if the distributed network is truly 

25 heterogenous, and there is overlapping management responsibil- 
ity among the sites, a more generalized solution involving 
distribution of the management director capabilities is 
desired. 

Some degree of distribution of management director 
30 capabilities is afforded by mechanisms' known in the art. For 
example, an interface between a user and a presentation module 
can be distributed geographically by using available remote 
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terminal connection mechanisms. But available mechanisms fall 
short of providing a generalized technique for use when 
management modules are distributed across different systems. 

One known mechanism of the prior art that has a 
5 limited capability to invoke a procedure remotely is the 
remote procedure call, or KPC. In the remote procedure call, 
a procedure is invoked in one system and executed in another 
system. Usually, the code that invokes the procedure has no 
"knowledge" that the procedure is actually executed in another 

10 system. The typical way in which the remote procedure call is 
implemented is that the name of the procedure is the key that 
determines which system will execute the procedure. As will 
become apparent from the detailed description that follows, 
this conventional remote procedure call technique is of little 

15 or no help in the context of the present invention. The 
invention is concerned with the problems inherent in invoking 
only two types of procedures remotely: a remote call to a 
function module and a remote call to an access module. As will 
also be better appreciated from the detailed description, the 

20 services that these remote calls provide are distinguished by 
various arguments associated with the call, rather than by the 
call name. Accordingly, the standard RPC technique cannot be 
used. 

In brief, the present invention provides a simple 
25 but effective solution to problems arising in distributed 
management of complex systems. 

SUMMARY OF THE INVENTION 

30 The present invention resides in apparatus, and a 

related method, for managing entities that may be distributed 
over multiple physical systems and multiple geographical 
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locations as generally recited in claims 1 and 6. The 
apparatus of the invention is embodied in a management system 
for monitoring or controlling at least one managed entity, 
which is a component of a complex and interactive collection 
5 of components. 

Briefly, and in general terms, the management system 
of the invention comprises at least one management module of 
a first type, referred to for convenience as a presentation 
module; at least one management module of a second type, 
10 referred to as a function module; at least one management 
module of a third type, referred to as an access module; and 
a management director kernel, for facilitating communication 
between a user of the system and one or more managed entities, 
through the management modules. A management module of the 
15 first type communicates between a user of the system and a 
management module of the second type, and a management module 
of the second type communicates between management modules of 
the first and third types, and performs a management function 
that requires forwarding of a user-initiated procedure call 
20 to a management module of the third type. A management module 
of the third type communicates the procedure call from a 
management module of the second type to a managed entity, and 
performs a desired function in relation to the managed entity. 

In the distributed arrangement with which the 
25 invention is concerned, the management modules of at least two 
of the first, second and third types are in different physical 
systems or at different locations, and the functions of the 
management director kernel may be performed by management 
director kernel subsystems at different locations. Further, 
30 the management director kernel includes "remote call processing 
means, for processing a procedure call that must be forwarded 
from a management module in one location to a management 



WO 93/20508 



PCT/US93/03402 



module in another location. 

The management director kernel includes remote call 
processing means, for processing a procedure call that must 
be forwarded from a management module in one location to a 
5 management module in another location. In accordance with one 
aspect of the invention, the remote call processing means 
includes remote access call processing means f for processing 
calls to invoke primitive functions in at least one management 
module of the third type, each such call being directed to the 

10 management of a single target entity. More specifically, the 
remote access call processing means includes means for 
determining for each call the identities of director kernel 
subsystems from which the target entity that is the subject 
of the call, can be managed; means for forwarding the call to 

15 a selected director kernel subsystem; and means in the 
selected director kernel subsystem, for forwarding the call 
through a management module of the third type, to the target 
entity to be managed. If the means for determining the identi- 
ties of director kernel subsystems from which the target 

20 entity can be managed determines that the target entity can 
be managed from anywhere, the remote access call processing 
means selects the current director kernel subsystem for 
management of the target entity. 

In accordance with another aspect of the invention, 

25 the remote call processing means includes remote function call 
processing means, for processing calls to invoke a higher- 
level function in at least one management module of the second 
type, each such call being executed in the context of a 
logical grouping of target entities, referred to as a domain. 

30 More specifically, the remote function' call processing means 
includes means for confirming that the call is made in the 
context of a domain; a domain database associated with the 
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management director kernel; means for determining from the 
domain database the identity of the director kernel subsystem 
that is responsible for the domain named in the call; means 
for forwarding the call to the director kernel subsystem 
5 responsible for the domain named in the call; means operative 
in the director kernel subsystem responsible for the domain/ 
for determining whether the call can be satisfied without 
invoking a call to another management module; and means 
operative in the director kernel subsystem responsible for the 

10 domain, for invoking a management module of the second type 
if the call cannot be satisfied without doing so, wherein the 
invoked management module of the second type performs a de- 
sired f unction in the context of the domain named in the call . 

In one illustrative form of the invention, the 

15 procedure call requests historical data pertaining to 
operation of the domain entities, and the remote function call 
processing means will obtain the historical data, if possible, 
directly .from a database located on the system that is 
responsible for the domain, and in which historical data is 

20 recorded, or will obtain the requested data by invoking an 
appropriate management module of the second type. In another 
illustrative form of the invention, the procedure call 
pertains to a background function being performed in relation 
to some or all of the entities belonging to the domain, and 

25 invokes a management module of the second type . 

In terms of a method or process, the invention 
comprises the steps of coordinating, in a management director 
kernel, communication of requests from a user, to control or 
monitor the managed entity, and the communication of response 

30 information back to the user; communicating, in at least one 
management module of a first type, between the user and a 
management module of a second type; communicating, in at least 
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one management module of a second type, user-initiated 
procedure calls between a management module or the first type 
and a management module of a third type; and communicating, 
in at least one management module of a third type, a user- 
5 initiated procedure call from a management module of the 
second type to a managed entity, to perform a desired function 
in relation to the managed entity. 

In management systems with which the invention is 
concerned, the management modules of at least two of the 

10 first, second and third types are at different locations and 
the functions of the management director kernel are performed 
by management director kernel subsystems at different loca- 
tions. The method of the invention includes processing at 
least one remote procedure call that must be forwarded from 

15 a management module in one location to a management module in 
another location. 

As described herein, the step of processing a remote 
procedure call includes processing a remote access call, to 
invoke a primitive function in at least one management module 

20 of the second type, each such call being directed to the 
management of a single target entity. The step of processing 
a remote access call includes determining for each call the 
identities of director kernel subsystems from which the target 
entity that is the subject of the call, can be managed; 

25 forwarding the call to a selected director kernel subsystem; 
and, in the selected director kernel subsystem, forwarding the 
call through a management module of the third type, to the 
target entity to be managed. If the step of determining the 
identities of director kernel subsystems from which the target 

30 entity can be managed determines that the target entity can 
be managed from anywhere, the step of processing a remote 
access call selects the current director kernel subsystem for 
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management of the target entity. 

Advantageously, the step of processing a remote 
procedure call includes processing a remote function call, to 
invoke, a higher-level function in at least one management 
5 module of the second type, each such call being executed in 
the context of a logical grouping of target entities, referred 
to as a domain. More specifically, processing a remote 
function call includes confirming that the call is made in the 
context of a domain; determining from a domain database 

10 associated with the management director kernel the identity 
of the director kernel subsystem that is responsible for the 
domain named in the call; forwarding the call to the director 
kernel subsystem responsible for the domain named in the call; 
determining, in the director kernel subsystem responsible for 

15 the domain, whether the call can be satisfied, such as by 
retrieving historical data, without invoking a call to another 
management module; and invoking, in the director kernel 
subsystem responsible for the domain, a management module of 
the second type if the call cannot be satisfied without doing 

20 so. The invoked management module of the second type performs 
a desired function in the context of the domain named in the 
call. 

As noted, earlier, the remote function call may be 
used to retrieve historical data relating to a domain of 

25 managed entities, or may relate to a background task being 
performed in relation to some or all of the entities belonging 
to the domain » 

It will be appreciated from the foregoing that the 
present invention represents a significant advance in the 

30 field of management of complex entities, such as distributed 
data processing systems. In particular, the invention provides 
a mechanism for passing procedure calls to* remotely located 
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management entities, consistent with a unified model of the 
management function. Two types of remote procedure calls are 
disclosed: a remote access call to invoke a primitive function 
of a single selected managed entity, and a remote function 
5 call to invoke a more complex function in the context of a 
domain of managed entities. In both cases , the mechanism of 
the invention forwards a procedure call to the appropriate 
management module, making use of management director kernels 
located in multiple physical systems . The remote function call 

10 made in the context of a domain provides the user with 
extremely useful domain-related data in a convenient and 
transparent manner. 

From the above summary, it will also be appreciated 
that the standard remote procedure call (RPG) of the prior art 

15 cannot be adapted to provide the same range of services as the 
present invention. The standard RPC provides services that are 
distinguishable from each other only by the use of different 
procedure names. In the present invention only two broad types 
of procedure are distinguishable by procedure name: the remote 

20 access call and the remote function call. But each type of 
call can be used to perform a variety of management opera- 
tions, defined by arguments of the procedure calls. The remote 
access call is invoked to perform a selected primitive 
function in any selected managed entity, and the remote 

25 function call is invoked to perform a selected higher-level 
function in the context of a domain of managed entities. 



30 



PRISF DggCMBffW QF THE PfiMJJNgg 

A more detailed understanding of the invention may 
be had from the following description of a preferred embodi- 
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ment, to be read and understood in conjunction with the 
accompanying drawing wherein: 

FIGURE 1 is a diagram showing the basic concept of 
5 management of a managed object through a management system; 

FIG. 2 is a block diagram depicting the components 
involved in managing a single object in more detail; 

FIG. 3 is a diagram showing the components of a 
management system, including management modules and a 
10 management director kernel; 

FIG. 4 is a block diagram also showing how the 
components of the management system interact from a control 
and/or data flow perspective; 

FIG. 5 is a block diagram of portions of a manage- 
15 ment system, similar in structure to FIG . 4 , but having 
management modules located in different physical systems; 

FIG. 6 is a flowchart showing the principal 
functions performed in processing a remote access call to a 
target device; and 
20 FIG. 7 is a flowchart showing the principal 

functions performed in processing a remote function call in 
the context of a domain. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

25 

As shown in the drawings for purposes of illustra- 
tion,, the apparatus and method of the present invention are 
concerned with the management of complex systems such as 
interconnected computer networks. A basic management model 
30 is shown in FIG. 1, and applies generally to the management 
of almost any entity. The model includes three major compo- 
nents: a managed object , indicated by reference numeral 10 , 
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10 



15 



20 



a management system 12 and a user 14. The managed object 10 
is any system or device that has multiple controllable or 
observable states. In the application with which the invention 
is principally concerned, the managed object may be a computer 
system, or a communication device, or a computer peripheral 
device such as a printer. The managed object 10 is either 
controlled by the management system 12, as indicated by line 
16, or is monitored by the management system, as indicated by 
line 18, or both. The management system 12, in turn, receives 
control signals from the user 14 and supplies data back to the 
operator, both by way of a user interface, illustrated as an 

operator terminal 20. 

Management of a single object is further illustrated 
in principle in FIG. 2, in which the management system 12 is 
termed a "director" and the managed object 10 is shown as 
being part of a managed "entity" 10' . The director 12 includes 
three types of modules: presentation modules, one of which is 
shown at 22, function modules, one of which is shown at 24, 
and access modules, one of which is shown at 26. The presenta- 
tion module 22 defines the appearance of management informa- 
tion presented by the director 12 to elements outside it, such 
as users or other management systems. The function module 24 
defines one or more services offered by management applica- 
tions residing within the director 12. The access module 26 
defines the mechanisms for control or monitoring of managed 

entities. . 

The shaded areas in FIG. 2 represent internal and 

external interfaces between various modules. It will be 

understood that the division of functions among the various 

modules and interfaces is, to some degree, a matter of design 

choice. Nevertheless, the illustrated director and entity 

models are useful generalizations with which to view the 
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process of management of complex systems. 

The foregoing description relates to the management 
of a single entity or object. A more realistic situation is 
one in which there are multiple managed objects, requiring 
5 multiple access modules , multiple function modules and, in 
general, multiple presentation modules. The director compo- 
nents are then something like those shown in FIG . 3 . Multiple 
presentation modules 22 f multiple function modules 24 and 
multiple access modules 26 are shown as being connectable to 
10 a director kernel 30 f comprising an interface section 32, and 
executive 34 and a management information repository 36. 

The interface section 32 and the executive 34 
provide an environment for management modules to exist and to 
interoperate. The management modules (22, 24, 26) can then be 
15 simply "plugged in" to the director kernel and can operate 
without specific a priori knowledge of each other. The 
management information repository (MIR) 36 defines the 
structure and storage of management information within a 
director. The MIR provides the means for specifying and 
20 storing information about managed entities and management 
module services. 

FIG, 4 shows the components of a management system 
viewed from a slightly different perspective, including 
multiple presentation modules 22, multiple function modules 
25 24 and multiple access modules 26. The director kernel or 
interface is indicated at 32A, connecting the presentation 
modules 22 to the function modules 24, and at 32B, connecting 
the function modules 24 to the access modules 26. The director 
kernel interface is shown as including three components: an 
30 information manager (40A, 4 OB), a dispatcher (42A, 42B) and 
a data storage module (44A, 44B) . It will be understood, 
however, that in a simple configuration the information 
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manager modules 4 OA, 40B in fact comprise a single computa- 
tional module, and that the dispatchers 42A, 42B may also 
comprise a single module residing on a single computational 
machine . 

5 Basically f the information manager 40A or 40B and 

the dispatcher 42A or 42B together process requests for 
communication between a management module on one level, such 
as a presentation module 22 , and a management module on the 
next lower level in the figure, such as a function module 24. 

10 A more specific description follows. 

If the information manager 4 OA receives a request 
from a presentation module 22 to which it can respond using 
the information in the data storage module 44A, it intercepts 
the request and generates a response to the request, which it 

15 transmits to an appropriate presentation module 22 for display 
to the operator who initiated the request. If the information 
manager 4 OA is unable to respond to the request, it then 
determines whether the request relates to the current time or 
a time in the future; that is, the information manager 4 OA 

20 determines whether the request should be processed immediately 
or scheduled for a specified time in the future. At the 
appropriate time, whether immediately or at the scheduled 
time, the information manager 4 OA transfers the request to the. 
dispatcher 42A. From the nature of the request, the dispatcher 

25 42A identifies a function module 24 to process the request, 
and transfers the request to that function module. 

In response to the receipt of a request from the 
dispatcher 42A, the function module 24 proceeds to process the 
request. It may, in response to the request, initiate one or 

30 more operations , each represented by a request, referred to 
as a subordinate request, which it directs to another function 
module or to the function-access interface 32B. Upon receiving 
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responses to all of the subordinate requests , the function 
module 24 generates a response f which it transmits to the 
dispatcher 42A. The dispatcher 42A then formulates a response, 
which it transmits , through the information manager 40A, to 
5 the appropriate presentation module 22 for display to an 
operator . 

Similarly > a subordinate request from a function 
module 24 , directed to the function-access interface 32B, is 
received initially by the information manager 40B- The data 

10 storage element 44B contains information , as provided by a 
historical data recorder function module , as to the condition 
of the complex system at various points in time, in particu- 
lar , selected information as to the status and conditions of 
the various entities controlled by the access modules 26. 

15 if the information manager 42B receives a subordi- 

nate request from a function module 24 to which it can respond 
using the information in the data storage element 44B, it 
intercepts the request and generates a response to the 
subordinate request ., which it transmits to the function module 

20 from which it received the subordinate request. If the 
information manager 4 OB is unable to respond to a subordinate 
request from a function module 24 f it then determines whether 
the request relates to the current time or a time in the 
future; that is, the information manager determines whether 

25 the request should be processed immediately or scheduled for 
a specified time in the future. At the appropriate time, 
whether immediately or at the scheduled time, the information 
manager 40B transfers the subordinate request to the dispatch- 
er 42B. In response to the receipt of a subordinate request 

30 from the information manager 40B, the dispatcher 42B identi- 
fies an access module 26 to process the subordinate request 
and transfers the subordinate request to that access module. 



\ 
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In response to the receipt of a subordinate request 
from the dispatcher 42B, the access module 26 proceeds to 
process the request. It may, in response to the subordinate 
request , initiate one or more operations in connection with 
5 the entity of the complex system controlled by the access 
module. If the subordinate request requires the access module 
26 to change the state or condition of the entity; it attempts 
to do so and generates a response containing status informa- 
tion indicating the status of the attempt, that is, for 

10 example, whether the change was successful, unsuccessful, or 
partially successful . On the other hand, if th<5 subordinate 
request requires the access module 26 to identify the state 
or condition of the entity, it generates a response indicating 
the entity's state or condition. Finally, if the subordinate 

15 request requires the access ■ dule 26 to do both, it attempts 
to change the state or condition of the entity and generates 
a response indicating the status of the attempt and also the 
entity's new state or condition. In any case, the access 
module 26 transmits the response to the dispatcher 42B, which 

20 transfers it to the function module .24 that generated the 
request. The function module 24 uses the response from the 
access module 26 in formulating its response to a request from 
the dispatcher 42B at the presentation-function interface, or 
to a subordinate request from another function module r as 

25 appropriate. A function module 24, upon receiving a subordi- 
nate request from other function modules, processes it in the 
same manner as it processes a request from the dispatcher 42B. 

The control arrangement depicted in FIG. 4 provides 
a number of advantages. Basically, it forms a processing 

30 chain, with each element along the chain attempting to process 
a request before passing it along to the next element. Thus, 
if the information manager 40A/B can process the request, 
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based on the contents of the associated data storage module 
44A/B, without requiring further processing by another element 
further down the chain, it does so. Furthermore, the control 
arrangement is extensible, so that additional presentation 
5 modules 22 , function modules 24 and access modules 26 can be 
easily added, without changing the architecture of the control 
arrangement . 

FIG. 5 is diagram that is similar to FIG. 4, in that 
the director kernel structure is depicted as comprising an 

10 information manager and a dispatcher, but different from FIG. 
4 in two other respects. First, only a single management chain 
is shown, involving one presentation module, one function 
module and one access module. Second, these three modules are 
shown as being distributed among different systems. That is 

15 to say, the three management modules are physically located 
on different machines and probably in different locations. 
Moreover, the management function depicted necessarily 
involves multiple management separate director kernels in 
different locations. 

20 More specifically, the presentation module is 

located in system #1 and is indicated by numeral 22(1). 
Similarly, the function module is located in system #2 and is 
indicated at 24 (2) , and the access module in located in system 
#3 and is indicated at 26 (3) . A request, otherwise known as 

25 a procedure call, originating in presentation module 22 (1) and 
invoking a procedure in function module 24 (2) must necessarily 
utilize the services of a director kernel in system #1 and a 
director kernel in system #2. In particular, the director 
kernel in system #1 employs an information manager 40A(1) and 
30 a dispatcher 42A(1) , which communicate with an information 
manager 40A(2) and dispatcher 42A(2) in system #2. The latter 
dispatcher selects function module 24(2) to perform the 
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requested procedure. Similarly , a procedure call originating 
in function module 24(2) employs an information manager 40B(2) 
and a dispatcher 42B(2) in system #2, which communicates with 
an information manager 40B(3) and dispatcher 42B(3) in system 
5 #3. Finally, dispatcher 42B(3) communicates the procedure to 
the access module 26(3) , which performs a requested function 
on an associated managed object (not shown in this figure) . 

The present invention is concerned with the imple- 
mentation of a remote procedure call in which the calling 

10 module and the called module are distributed across different 
physical systems , and may be distributed geographically. As 
can be seen from FIG. 5, there are two possible levels of 
remote calling when management modules are distributed. One 
is a remote call to invoke what might best be termed a 

15 "primitive" service, typically located in an access module. 
This type of call or request will be referred to in this 
specification as a "remote access call." The service provided 
by an access module is typically "primitive" in the sense that 
it might involve monitoring the current status of a managed 

20 object, or changing some parameter in the object, or switching 
it on or off. By way of contrast, the other type of remote 
call or request is to invoke what might be termed a "value- 
added" service, typically located in a function module. For 
example, a value-added request might inquire as to historical 

25 data concerning operation of the managed object or objects. 
This type of remote call will be referred to as a "remote 
fraction call." 

It is worth noting that, in fact, from a user 
standpoint, there is no special remote function call or 

30 special remote access call. In keeping with the generalized 
management model described above and in the cross-referenced 
application, a called management module is unaware of whether 
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it is being called "remotely" front a different system, or 
whether it is being called "locally" from the same system. 
Similarly , a calling management module need have no concern 
as to the location of the called module. The access calls and 
5 function calls are generalized to handle both local and remote 
calls , but the focus of the present explanation will be on the 
manner in which remote procedure calls are handled in the 
director kernel and management modules. 

Typically, a remote access call is invoked by a 

10 function module 24 when it cannot satisfy a procedure call 
made to it without routing the call to an access module 26. 
The parameters of a remote access call include the identity 
of the management operation to be -performed by the access 
module 26, and the name of the managed entity on which the 

15 management operation is to be performed. Again f this should 
be distinguished from a conventional remote procedure call 
(RPC) , in which the management operation is specified by the 
procedure name. There are three possible situations involving 
remote access to a managed entity: 

20 (a) The managed entity can be accessed from only one 

system. For example, if the managed entity is a modem or other 
piece of hardware connected to a specific input/output port 
of a computer system, there is simply no other access route 
to the entity. 

25 (b) The managed entity can be accessed from a 

limited number of different systems. 

(c) The managed entity can be accessed from any 

system. 

If the situation is as described in (c) , the entity 
30 is universally accessible from any system, and the concept of 
remote access is part of the normal management protocol 
interactions between the access module and the entity. If 
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access is limited to one system, as in (a) , or a number of 
systems, as in (b) , it is necessary to associate with the 
managed entity a list of management directors that can access 
the entity. A mechanism is already available in the basic 

5 management model for taking care of this. The mechanism is 
known as the distributed name service (DNS) . The name of each 
managed entity, and other information relating to the device, 
is stored in a universally accessible data base. To effect a 
remote access call, it is only necessary to add to the DNS 

0 some further information about each managed entity. Specifi- 
cally, the DNS entry for each managed entity must also contain 
a list of the names of managing directors, if any, that can 
access the entity. If the list is empty, this is a default 
condition indicating that the entity may be accessed from 

5 anywhere. The mechanism for entering this information in the 
DNS is a management function module known as the registration 
function module. (In the cross-referenced application, these 
functions were described as being performed by a management 
function module referred to as the configuration function 

0 module . ) 

The basic steps followed by a function module in 
calling an access module, which may be remotely located, are 
shown in FIG. 6. The first step, as indicated in block 50, is 
to obtain information on the target entity or device, using 

5 the distributed name service (DNS) . Included in this informa- 
tion are the names of any directors from which the target 
entity can be accessed. If no director names are found, as 
determined in block 52, the target entity is accessible from 
anywhere, and remote access is not used. An access module is 

0 selected within the director from which a procedure call is 
to be initiated, as indicated in block 54, and the call is 
completed in a normal manner, as indicated in block 56. This 
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may include handling a response from the selected access 
module. 

If the target entity must be accessed from another 
director, other than the one from which the procedure call is 
5 being initiated, the next step is to select a director name 
from the list obtained (in steps 50 and 52) , and to determine 
the system on which the selected director is located, as 
indicated in block 58. Then an access module capable of 
performing the desired function on the target entity is 

10 selected, as indicated in block 60. Basically, the call is 
passed or forwarded to the selected access module, such as 
access module 26(3) in FIG. 5, by way of a chain of distribut- 
ed director components, such as information manager 40B(2) , 
dispatcher 40B(2) , information manager 40B(3) , and dispatcher 

15 42B(3) . Any response data associated with the call is passed 
back through the same distributed director chain. 

The step of selecting a director, indicated in block 
58, from a: list of directors obtained through the distributed 
name service (DNS) may include selecting a second director if 

20 the first one selected is currently busy or unavailable for 
some other reason. If communication cannot be established with 
a selected director, another choice is made from the directors 
named as being capable of accessing the desired entity. If the 
entity cannot be reached through any of the listed directors, 

25 an error indication is provided to the calling module. 

The remote function call is made to invoke a "value- 
added" function, beyond the capabilities of an access module, 
and usually performed by a function module. A significant 
feature of the remote call function is that it may be made in 

30 the context of a "domain" of managed entities, rather than in 
the context of a single target entity, as in the case of a 
remote access call. A domain is a user-defined logical 
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collection of managed entities. A user managing a complex 
system may, for example, be interested in historical perfor- 
mance data relating to all of the network terminals in a 
particular building, and for this purpose the user would 
5 define these terminals as belonging to a single domain. In 
other instances, it may be convenient to define separate 
domains 7or the entities, associated with various departments 
within a corporate structure, or to define different domains 
for different equipment types. For some purposes, it may be 

10 desirable to define a domain as including other domains. 

A domain database is maintained to keep track of the 
membership in each domain, as well as the name of the director 
responsible for each domain, and other information relating 
to each domain. In processing a function call, i.e. a call to 

15 a function module, a director first determines whether the 
call is made in the context of a domain, as indicated in block 
72 of FIG. 7. If the call parameters do not identify it as one 
made in the context of a domain, the call is handled as a 
function call within the calling system, as indicated in block 

20 74, and as described in the cross-referenced patent applica- 
tion. If the call is made in the context of a domain, the 
domain database is accessed (block 76) , and the name of the 
director responsible for the domain named in the call is 
obtained (block 78) . Then the call is forwarded to the 

25 director responsible for the domain (block 80) . Once the call 
is being processed by the director responsible for the domain 
with which the call is concerned, the next step is to 
determine, as indicated in block 82, whether the call can be 
satisfied without invoking a function module. The remote 

30 function call may, for example be a request for historical 
data pertaining to members of the domain. Compilations of 
historical data are typically stored in the data storage 
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modules 44 associated with the management directors. If the 
call or request can be satisfied by retrieving data from the 
storage module 44 associated with the director responsible for 
the domain, then this action is taken, as indicated in block 
5 84* Then the call is completed in a normal manner, as 
indicated in block 86. If the call cannot be satisfied without 
invoking a function call, then an appropriate function module 
is invoked, as indicated in block 88, and the call is 
completed. Satisfying the call may require invoking one or 

10 more access modules to obtain requested raw data. 

An advantage of the domain-based function call is 
that it permits grouping of data by domain for presentation 
to a user. Domain-based function calls may also be made to 
request that historical data be stored automatically for each 

15 member of the domain, or that background processing be 
performed for each member of the domain. An example of 
background processing is a continual check on device status 
and alarm reporting in the event of unusual conditions. 

It will be appreciated from the foregoing that the 

20 present invention represents a significant advance in the 
field of management of complex systems. In particular, the 
invention provides a technique for handling management 
functions in cases where components of the management system 
may be distributed among different physical systems. The 

25 invention also provides for the convenient collection and 
presentation of data in the context of user-defined domains 
of managed entities- It will also be appreciated that, 
although an embodiment of the invention has been described in 
detail for purposes of illustration, various modifications may 

30 be made without departing from the spirit and scope of the 
invention. 
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Bffegt of the lavtmtipa 

A distributed digital data processing system is 
described herein. Network management basically consists in 
5 monitoring and control- A management entity or director 
comprises a presentation module (22) , a function module (24) 
and an access module (26) . Sometimes a need arises for 
geographic dispersion of the modules. In prior art, what is 
known as RPC or remote procedure call is known. In the prior 

10 art RPC f a procedure is invoked in one system and executed in 
another system wherein, the code that invokes the procedure 
may not have any knowledge that the procedure is actually 
executed in another system . Such an RI?C will be of no use in 
the present invention where only two types of procedures are 

15 used, which are (i) a remote call to a function module and 
(ii) a remote call to an access module. 

The present invention provides apparatus and method for 
effecting distrubted management of complex systems. 

The system described herein has management modules of 

20 three types: the presentation module, function module and the 
access module, wherein at least two of the moduels are at 
different locations. The management director kernel includes 
multiple subsystems at different locations, and includes a 
remote call processing means for processing a procedure call 

25 which needs to be forwarded from a management module in one 
location to that in another. 
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CLAIMS 

We claim: 

1 1. A management system (12), for monitoring or 

2 controlling at least one managed entity (10) which is a 

3 component of a complex and interactive collection of compo- 

4 nents f the system comprising: 

5 at least one management module of a first type (22) ; 

6 at least one management module of a second type (24) 

7 at least one management module of a third type (26) ; 

8 and 

9 a management director kernel (32A) , for facilitating 

10 communication between a user of the system and one or more 

11 managed entities , through the management modules; 

12 wherein a management module of the first type (22) 

13 communicates between a user of the system and a management 

14 module of the second type; 

15 and wherein a management module of the second type 

16 (24) communicates between management modules of the first and 

17 third types r and performs a management function that requires 

18 forwarding of a user-initiated procedure call to a management 

19 module of the third type; 

20 and wherein a management module of the third type 

21 (26) communicates the procedure call from a management module 

22 of the second type (24) to a managed entity, and performs a 

23 desired function in relation to the managed entity; 

24 and wherein the management modules of at least two 

25 of the first, second and third types are at different 

26 locations and the functions of the management director kernel 

27 includes multiple management director kernel subsystems at 

28 different locations; 
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29 and wherein the management director kernel includes 

30 remote call processing means (40) (42), for processing a 

31 procedure call that must be forwarded from a management module 

32 in one location to a management module in another location. 

1 2. A management system as defined in claim 1, 

2 wherein : 

3 the remote call processing means includes remote 

4 access call processing means, for processing calls to invoke 

5 primitive functions in at least one management module of the 

6 third type, each such call being directed to the management 

7 of a single target entity. 

1 3. A management system as defined in claim 2, 

2 wherein the remote access call processing means includes: 

3 means for determining for each call the identities 

4 of director kernel subsystems from which the target entity 

5 that is the subject of the call, can be managed; 

6 means for forwarding the call to a selected director 

7 kernel subsystem; and 

8 means in the selected director kernel subsystem, for 

9 forwarding the call through a management module of the third 

10 type, to the target entity to be managed, wherein 

11 if the means for determining the identities of 

12 director kernel subsystems from which the target entity can 

13 be managed determines that the target entity can be managed 

14 from anywhere, the remote access call processing means selects 

15 the current director kernel subsystem for management of the 

16 target entity. 

1 4. A management system as defined in claim 1, 

2 wherein: 
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3 the remote call processing means includes remote 

4 function call processing means , for processing calls to invoke 

5 a higher-level function in at least one management module of 

6 the second type, each such call being executed in the context 

7 of a logical grouping of target entities , referred to as a 

8 domain • 

1 5. K management system as defined in claim 5, 

2 wherein the remote function call processing means includes: 

3 means for confirming that the call is made in the 

4 context of a domain; 

5 a domain database associated with the management 

6 director kernel; 

7 means for determining from the domain database the 

8 identity of the director kernel subsystem that is responsible 

9 for the domain named in the call; 

10 means for forwarding the call to the director kernel 

11 subsystem responsible for the domain named in the call; 

12 means operative in the director kernel subsystem 

13 responsible for the domain, for determining whether the call 

14 can be satisfied without invoking a call to another management 

15 module; and 

16 means operative in the director kernel subsystem 

17 responsible for the domain, for invoking a management module 

18 of the second type if the call cannot be satisfied without 

19 doing so, wherein the invoked management module of the second 

20 type performs a desired function in relation to the domain 

21 named in the call, wherein 

22 the procedure call requests historical data 

23 pertaining to operation of the domain entities; and 

24 the remote function call processing means will 

25 obtain the historical data, if possible, directly from a 
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26 database in which historical data is recorded, or will obtain 

27 the requested data by invoking an appropriate management 

28 module of the second type. 

1 6 . A management method for monitoring or controlling 

2 at least one managed entity, which is a component of a complex 

3 and interactive collection of components, the method compris- 

4 ing the steps of : 

5 in a management director kernel, coordinating 

6 communication of requests from a user, to control or monitor 

7 the managed entity, and the communication of response 

8 information back to the user; 

9 in at least one management module of a first type, 

10 communicating between the user and a management module of a 

11 second type; 

12 in at least one management module of a second type, 

13 communicating" user- initiated procedure calls between a 

14 management module or the first type and a management module 

15 of a third type; and 

16 in at least one management module of a third type, 

17 communicating a user-initiated procedure call from a manage- 

18 ment module of the second type to a managed entity, to perform 

19 a desired function in relation to the managed entity; 

20 wherein the management modules of at least two of 

21 the first, second and third types are at different locations 

22 and the functions of the management director kernel are 

23 performed by management director kernel subsystems at 

24 different locations; 

25 and wherein the method further includes processing 

26 at least one remote procedure call that must be forwarded from 

27 a management module in one location to a management module in 

28 another location. 
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1 7. A management method as defined in claim 6, 

2 wherein r 

3 the step of processing a remote procedure call 

4 includes processing a remote access call, to invoke a 

5 primitive function in at least one management module of the 

6 third type, each such call being directed to the management 

7 of a single target entity. 

1 8 » A management method as defined in claim 7, 

2 wherein the step of processing a remote access call includes: 

3 determining for each call the identities of director 

4 kernel subsystems from which the target entity that is the 

5 subject of the call, can be managed; 

6 forwarding the call to a selected director kernel 

7 subsystem; and, 

8 in the selected director kernel subsystem, f orward- 

9 ing the call through a management module of the third type, 

10 to the target entity to be managed, wherein: 

11 if the step of determining the identities of 

12 director kernel subsystems from which the target entity can 

13 be managed determines that the target entity can be managed 

14 from anywhere, the step of processing a remote access call 

15 selects the current director kernel subsystem for management 

16 of the target entity. 



1 9. A management method as defined in claim 6, 

2 wherein: 

3 the step of processing a remote procedure call in- 

4 eludes processing a remote function call, to invoke a higher- 

5 level function in at least one management module of the second 

6 type, each such call being executed in the context of a 
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7 logical grouping of target entities, referred to as a domain, 

8 wherein the step of processing a remote function 

9 call includes: 

10 confirming that the call is made in the context of 

11 a domain; 

12 determining from a domain database associated with 

13 the management director kernel the identity of the director 

14 kernel subsystem that is responsible for the domain named in 

15 the call; 

16 forwarding the call to the director kernel subsystem 

17 responsible for the domain named in the call; 

18 in the director kernel subsystem responsible for the 

19 domain, determining whether the call can be satisfied without 

20 invoking a call to another management module; and 

21 in the director kernel subsystem responsible for the 

22 domain, invoking a management module of the second type if the 

23 call cannot be satisfied without doing so, wherein the invoked 

24 management module of the second type performs a desired 

25 function in the context of the domain named in the call. 
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